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Abstract 

We present Integer Linear Programming (ILP) Modulo Theories 
(IMT). An IMT instance is an Integer Linear Programming in- 
stance, where some symbols have interpretations in background 
theories. We develop the theoretical underpinnings for IMT by 
means of BC(r), the Branch and Cut Modulo T abstract transi- 
tion system. We show that BC(r) provides a sound and complete 
optimization procedure for the ILP Modulo T problem, as long as 
T is a decidable, stably-infinite theory. 

Our motivation for developing IMT is to enable advances in 
synthesis by coupling high-level declarative languages with new, 
powerful constraint solving and optimization techniques. This al- 
lows designers to specify what they want, not how to achieve it. We 
have applied the IMT approach to industrial synthesis and design 
problems with hard real-time constraints arising in the development 
of the Boeing 787. Given that many practical problems ranging 
from operations research to software design can be thought of as 
synthesis problems, we believe that our framework has the poten- 
tial to be widely applicable. 

General Terms Design, Languages 

Keywords Declarative Programming, Constraint Programming, 
Decision Procedures, Satisfiability Modulo Theories, Linear Pro- 
gramming, Optimization, Extensible Languages 

1. Introduction 

Managing the complexity of modem software-intensive systems 
requires raising the level of discourse by utilizing abstraction, in 
particular high-level modeling. For example, Boeing's 787 Dream- 
liner consists of thousands of interacting components, and just the 
software enabling fly-by- wire control is over 10 million lines of 
code. To design such complex systems, numerous models at var- 
ious levels of abstraction are used. The highest level models are 
architectural models; they describe structural properties and com- 
ponent interactions. While many architecture description languages 
have been proposed, these languages require users to specify what 
components are to be used and how they are to be connected. The 
effort required to do this can be significant. According to Boeing 
engineers, during the design of the 787 Dreamliner, the construc- 
tion of the architectural models required the "cooperation of multi- 
ple teams of engineers working over long periods of time." 
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Recently, we developed the CoBaSA (Component-Based System 
Assembly) framework. Using CoBaSA, we algorithmically syn- 
thesized architectural models using the actual production design 
data and constraints arising during the development of the Dream- 
liner 1 19 29 1. CoBaSA provides a high-level modeling and specifi- 
cation language, coupled with a synthesis tool based on verification 
and optimization technology. Key to the success of our approach 
was the combination of ILP with a custom decision procedure for 
real-time scheduling. We were able to automatically synthesize 
architectures in minutes directly from the high-level requirements. 
This allows designers to specify what they want, not how to achieve 
it. It enables deep design-space analysis and exploration at the ear- 
liest stages of design, when this kind of analysis is most useful. 
Finally, our work makes it possible to analyze and respond to dis- 
ruptive events, such as changes to requirements, in real time. 

In many ways, Boeing's 787 Dreamliner is one of the most com- 
plex systems ever built, and our success in fully automating signif- 
icant, industrially-relevant design problems is compelling evidence 
of the potential advances in synthesis that new constraint solving 
techniques can enable. To be more broadly successful, new ap- 
proaches to synthesis should be both expressive enough to model 
design problems arising in a variety of real systems and scal- 
able enough to automatically synthesize solutions. It is difficult 
to make the case with a theoretical argument, since synthesis is 
hard (NP-Hard) even with very simple classes of constraints, and 
very quickly becomes undecidable |25 1. In this paper, we show that 
while our framework has been primarily used for component-based 
design, it can also be applied to different problems in fields varying 
from operations research to database design. In fact, CoBaSA has 
been designed with extensibility in mind and allows for domain- 
specific modeling constructs. CoBaSA is publicly available^ 

To be widely applicable, we desire a parametric framework that 
can be instantiated for the particular class of synthesis problems. To 
this end, we propose the Integer Linear Programming (ILP) Modulo 
Theories (IMT) framework for combining ILP with background 
theory solvers. We can use Mixed-Integer Programming (MIP) 
instead of ILP, but in this paper we will focus on ILP for simplicity. 
By relying on ILP as the core, we build upon more than five decades 
of intensive research in integer programming |17|. In addition, 
the ILP emphasis on optimization (as opposed to feasibility) is 
of fundamental importance in synthesis, as solution spaces are 
typically very large, and the goal is to find solutions that are optimal 
with respect to certain design objectives. 

The primary goal of this paper is to introduce the theoretical un- 
derpinnings of IMT. We do that via the BC(r) framework (Branch 
and Cut Modulo T). BC(r) models the branch-and-cut family of 
algorithms for integer programming as an abstract transition sys- 
tem, and allows plugging in theory solvers. Building on classical 
results on combining decision procedures f26i 14 I I. we show that 



http : //www . CCS . neu . edu/home/vpap/cobasa . html 



1 



2012/10/16 



BC(T) provides a sound and complete optimization procedure for 
tlie combination of ILP witli stably-infinite theories. 

BC(T) is the IMT counterpart to the DPLL(T) architecture for 
lazy SMT 1 36 1. We study the relationship between the two systems 
and show that BC(r) can simulate DPLL(r), i.e., a BC(r)-based 
solver can be used as a drop-in replacement for an SMT solver that 
supports Quantifier-Free Linear Integer Arithmetic (QF_LIA). 

Contributions We provide the first to our knowledge frame- 
work for combining ILP with theories (IMT). We provide a gen- 
eral BC(r) architecture for solving IMT problems. We prove that 
BC(T) provides a sound and complete optimization procedure for 
the combination of ILP with a stably-infinite theory T. 

Paper Structure The rest of the paper is organized as follows. 
Section[2]introduces the CoBaSA synthesis paradigm by means of 
examples. In Section [3] we formally define IMT and provide an 
abstract BC(r) architecture for solving IMT problems. Section p] 
elaborates on the relationship between SMT and IMT. Section p] 
contains experimental evidence that the combination of ILP and 
theories is crucial for the practicality of CoBaSA. We provide an 
overview of related work in Section[6] and conclude with Section^ 

2. Synthesis Paradigm 

CoBaSA was initially developed to tackle problems that arise in 
the development of large-scale, component-based systems. The en- 
gineers building a large-scale system, e.g., a commercial airplane, 
have access to a pool of components. Their job is to select some 
of these components, connect, integrate, and assemble them in a 
meaningful way. We call this the system assembly problem. The 
CoBaSA modeling language provides object-oriented constructs to 
describe the components of a system, and also declarative ways 
to impose constraints on how these components will inter-operate. 
The synthesis backend is responsible for finding a way to select 
and integrate the components in a way that meets the high-level 
specification. 

Our modeling language and constraint solving engine are gen- 
eral enough to be useful in other classes of applications, not nec- 
essarily involving component-based systems. In this section, we 
will provide an introduction to the CoBaSA language by means of 
examples arising in different domains. Our examples demonstrate 
how the system can be augmented for domain-specific needs. We 
introduce CoBaSA concepts on demand. 

2.1 Applications 
2.1.1 TA Allocation 

We are given a pool of teaching assistants (TAs), a set of instructors, 
and a set of courses. Instructors have positive preferences, e.g., 
they frequently want to work with their advisees. They also have 
negative preferences, as they may have negative impressions of 
some TAs. Each course requires a fixed number of TAs. We have to 
determine which course each TA will participate in. 

The CoBaSA language is typed. The basic datatypes provided 
are integers (optionally bounded), Booleans, and strings. Entities 
resemble classes in an object-oriented language. The entity defini- 
tions of Figure[T]reflect the problem setup. Instructor and TA enti- 
ties have a single "id" field (which can be their name). Each course 
has an integer as the identifier (course number), a name, a list of 
instructors, and a bounded integer for the required number of TAs 
(field needs). A course also carries a list of instructor preferences, 
for which we need a helper entity. The idea is that instructors assign 
points to denote preferences, with the most desirable TA getting the 
maximum number of points. 

With entity definitions in place, we proceed to describe the 
actual data (Figure [2](. We model two of the instructors in our 



entity instructor { 
id: string 

>; 

entity ta { 
id: string 

}■; 

entity preference ■[ 
ta: ta; 

points : bomided 1 to 6 

y-, 

entity course { 
id : int ; 
name: string; 

instructors: list instructor; 
needs: bounded to 10; 
preferences: list preference 

>; 

Figure 1 : TA Allocation Entities 

var manolios = instructor{"Pete Manolios"}; 

var wahl = instructor{ "Thomas Wahl"}; 

var ramachandran = ta-["Jaideep Ramachandran"} ; 

var cs2800 = 
course-[ 

2800; "Logic and Computation"; 

[manolios; wahl]; 4; 

[pref erence{ramachandran; 6}-; ...] 

}; 

var tas = [ramachandran ; ...]; 

var courses = [cs2800; ...]; 

Figure 2: TA Allocation Data 

department, Manolios and Wahl. We also instantiate the ta entity 
for Ramachandran. We are then ready to describe the "Logic and 
Computation" course (CS2800), taught by Manolios and Wahl. 
CS2800 requires four TAs. The instructors do their best to get 
Ramachandran in the team, i.e., he gets 6 preference points. We 
group TAs and courses in lists. 

map m tas courses [] [] ; 

forall c in courses: 

c. needs = sum ta in tas: m(ta, c) ; 

min -(sum c in courses: 

sum p in c .pref erences : 
m(p.ta, c) * p. points); 

Figure 3: Defining a map from TAs to Courses 

Our goal is to assign exactly one course to each TA. We achieve 
this with the CoBaSA map statement (Figure |3|. map existentially 
defines a function whose domain is the list of TAs and whose range 
is the list of courses. We subsequently constrain the mapping so that 
each class will get the required number of TAs. The sub-language 
for expressing constraints supports arbitrary Boolean combinations 
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of linear arithmetic constraints. It also provides facilities for rang- 
ing over all elements of a list, either for succinctly applying a con- 
straint to all elements (forall), or for adding up an expression 
(sum). The map reference m(ta, c) is true iff ta is mapped to c. 
It is valid for a Boolean expression like m(ta, c) to appear within 
an arithmetic expression, behaving as a {0, 1} -integer. 

CoBaSA is built on top of optimization technology, and thus 
allows programmers to provide objective functions. For TA alloca- 
tion, our objective is to keep instructors as happy as possible. The 
objective function of Figure |3] maximizes the sum of preference 
points for the TA assignments that actually match the instructors' 
wishes (we maximize a quantity by minimizing its negation). We 
sum over all courses, and over all elements of the preference list 
associated with each course. 

The problem is easy to tackle with CoBaSA: the language pro- 
vides the right abstractions and the ILP solver has no trouble find- 
ing an optimal solution quickly. We have used CoBaSA success- 
fully to find a TA allocation for the Spring 2012 semester in our de- 
partment. The requirements took less than a person-hour to encode. 
CPLEX |2| needed less than a second to find an optimal solution. 

2.1.2 Database Debugging 



var output = 

y, select X from employees 

where x.age <= 35 and x. salary >= 3500; 

(a) Query 

'/. exists X in output: true; 

(b) Table Constraint 

var limit: bounded to 120; 

var output = 

7, select X from employees 

where x.age <= limit and x. salary >= 3500; 

min limit; 

(c) Query with Symbolic Values 
Figure 5: Domain-specific Syntax for Database Queries 



id 


age 


salary 





32 


3000 


1 


28 


2800 


2 


30 


3200 


3 


41 


4500 


4 


X 


y 



entity emp_schema -[ 
id: int; 

age: bounded to 120; 
salary: bounded to 20000 

}; 

var x: bounded to 120; 
var y: bounded to 20000; 



var employees = 

[emp_schema{0; 32; 3000> 

emp_schema{l; 28; 2800> 

emp_schema{2; 30; 3200} 

emp_schema{3; 41; 4500> 

emp_schema{4; x; y}] ; 



Figure 4: Database Table Representation 

Database users frequently encounter the following scenario: 
they perform a query, but the output table does not contain some 
of the expected results. Sometimes users even get no output tuples 
at all. 

Constraint solving is a good fit for debugging problematic 
queries. If the output of a query falsifies a given specification, we 
can try to synthesize one or more additional input tuples that trans- 
late to output tuples with the desired characteristics. By providing 
such tuples we prove that the query is not altogether inconsistent. 

We show how to embed database concepts in CoBaSA. As a 
first step, we want to be able to describe database tables. A table 
schema (i.e., the field names and types of a table) can be sufficiently 
described with a CoBaSA entity definition. Tuples in a certain table 
are encoded as instances of the appropriate table schema; tables are 
just lists of instances. Figure|4]demonstrates how we can represent 
a table of employees. Note that the table contains symbolic data, in 
particular an employee with unknown age and salary. 

Describing database queries is harder. Instead of trying to ex- 
press queries in terms of the core language constructs, we utilize 
the framework's syntax extension mechanism to define a Domain- 
Specific Language (DSL) that provides database operators like se- 
lection, projection, union, join, and cross-product. The database 
query of Figure |5a| now becomes valid CoBaSA syntax (we pro- 
duce a table named output with all employees aged at most 35 
whose salary is at least $3500). Note that the operators we define 



do not just perform evaluation with concrete values (like a DBMS 
would), but work with a mix of concrete and symbolic values. We 
then impose a constraint on the output table (Figure[5b}: in our sim- 
ple example, the output table shouldn't be empty. CoBaSA looks 
for appropriate input values for the record with id 4 so that the re- 
quirements will be met: it comes up with an employee who is 35 
years old and earns $3500 a month. 

Instead of coming up with new tuples, the user may want us to 
fix the query itself, i.e., synthesize a query that is reasonably close 
to the original, but produces the expected answer. For example, we 
can relax some conditions so that more tuples will go through. In 
our example, we could let CoBaSA find a different age limit. In 
Figure |5c] we achieve this by providing a query skeleton, which 
will turn into a concrete query once we find a value for limit. 
Optimization capabilities are crucial, because showing that 120 is 
an appropriate age limit would be trivial but not useful. 

val plugin_select : 

id -> texpr -> texpr -> env -> texpr 

let extend e = 
EXTEND 
e: FIRST 

[ [ "•/."; "select"; i = LIDENT; 
"from"; el = SELF; 
"where"; e2 = SELF -> 
E_Ext (plugin_select i el e2) ] 

] ; 
END 

Figure 6: Anatomy of the Syntax Extension 

To allow embedding DSLs, we build upon Camlp5 Extensible 
Grammars fTl. Figure[6]gives a glimpse of how we actually extend 
the CoBaSA grammar for databases. The non-terminal symbol for 
expressions (e) is extended with a new production for database 
selection. We define the semantics of this extension by means of a 
compiler "plugin", the function plugin_select. In the fragment 
of code we show, plugin_select is partially applied: its last 
argument is the environment. When this closure is finally applied 
on the appropriate environment argument, it will produce a term 
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in the internal representation of expressions (type texpr), i.e., 
plugin_select desugars the select operator. 



2.1.3 Architectural Synthesis with Real-Time Constraints 

We applied CoBaSA to solve the architectural synthesis prob- 
lem for the real, production data from the 787 Dreamliner, which 
was provided to us by Boeing 1 19 |. The basic components for this 
problem are anywhere from 8 to 22 cabinets (providing resources 
like CPU time, bandwidth, battery backup, and memory), about 200 
software applications that reside on the cabinets (and consume re- 
sources), and up to 300 global memory spaces (that also consume 
resources). Applications communicate via a publish-subscribe net- 
work where messages are aggregated into virtual links and trans- 
mitted via multicast. 

The core CoBaSA language suffices to model all classes of re- 
source and architectural requirements. For illustration purposes, we 
provide a scaled-down example (Figure |7j that retains the flavor 
of the actual models. We focus on applications and cabinets, and 
omit other kinds of components; we also ignore resources other 
than RAM and CPU time. We start by defining the entities (Fig- 
ure |7a|(. We then proceed to instantiate the entities. We have four 
concrete cabinets, each of which provides 256MB of RAM and 
1000000 CPU cycles per second. We also have 8 applications. We 
subsequently organize applications and cabinets in lists (Figure[7c). 

Each application has to be mapped to (i.e., reside on) exactly 
one cabinet (Figure |7d[ l. The mapping is subject to resource re- 
quirements: the applications that co-reside on a cabinet cannot ex- 
ceed the provided amount of CPU and RAM. The map statement 
provides a shorthand for expressing such resource constraints. The 
map m is further constrained to satisfy other (i.e., non-resource) 
requirements. For example, applications t7_0 and t7_l are two 
copies of the same safety-critical process; they have to reside on 
different cabinets to achieve fault-tolerance. This is a separation 
constraint, and can be expressed as shown in Figure |7e] Similarly, 
we can have co-location constraints, components fixed on specific 
cabinets, etc. 

Interesting architectural synthesis instances from the aerospace 
domain also have optimization criteria. For example, we may want 
to minimize the maximum bandwidth consumption across cabi- 
nets, i.e., balance the load. The rationale is that the requirements 
may change in the future (e.g., a component supplier is unable 
to meet the agreed-on resource consumption), but it is not always 
possible to change the design, for instance because the cost of re- 
certification is astronomical. When this happens, some headroom 
in resource availability is precious. 

The architectural models we synthesize also have to satisfy hard 
real-time constraints. The scheduling discipline (static cyclic) is 
non-preemptive and very hard to satisfy in practice. This class of 
constraints is not easy to model with the core CoBaSA language. In 
fact, the problem is deeper, as we do not know how to reduce static 
cyclic scheduling constraints to ILP constraints in a way that leads 
to reasonable solving times by modern ILP solvers. Therefore, we 
cannot follow the approach of Section [2. 1.2[ i.e., compile higher 
modeling constructs to the core constructs, without touching the 
underlying formalism. 

Instead what we need is an extension to the underlying logic 
(integer linear constraints) and the corresponding solver We have 
implemented a custom decision procedure for scheduling, and a 
mechanism for this decision procedure to communicate with the 
ILP solver |19|. The backend now interleaves ILP solving and 
scheduling, i.e., we have an instance of the ILP Modulo Theories 
framework. In each iteration, CoBaSA obtains a satisfying assign- 
ment for an ILP instance. If the assignment is consistent with the 
scheduling requirements, CoBaSA reports an answer. Otherwise, 



CoBaSA learns a theory lemma that precludes the unschedulable 
allocation, along with a class of similar unschedulable allocations. 

The backend extension gives rise to a schedulability predi- 
cate that appears in the models: schedCm, tasks, cabinets) 
(where m is a map and tasks, cabinets are lists) is true iff the 
tasks co-located on each cabinet according to the map m are schedu- 
lable. In other words, extending the underlying formalism is re- 
flected on the modeling language. 

2.2 Discussion 

We described different applications of the CoBaSA language. All 
three applications give raise to integer linear constraints. In all 
cases optimization is required, or at least strongly desired. Integer 
arithmetic and optimization requirements point to ILP as a suitable 
backend for synthesis. We experimentally show that ILP indeed 
works better than the alternatives in Section|5] 

The applications differ with respect to their language require- 
ments, (a) For TA allocation, core CoBaSA suffices, (b) Database 
debugging requires domain-specific syntax, (c) For architectural 
synthesis, we extend the backend with a custom decision pro- 
cedure. It is possible for an application to require extensions in 
both directions. For example, we can have an aerospace DSL with 
separation and co-location operators (among others), combined 
with a background theory for scheduling. The separation con- 
straint shown in Figure |7e] can then be written more succinctly 
as Sep (m, t7_0, t7_l). 

From a decision procedure perspective, extending the language 
without touching the ILP backend resembles the eager approach to 
SMT 1 16l l28ll40l , i.e., compiling high-level constructs like bitvec- 
tor operations to SAT. Coupling an ILP solver with an external de- 
cision procedure resembles the lazy SMT architecture fF, '11! '36|. 
Which of the two approaches is better depends on the problem 
structure and size. The lazy approach is harder to implement, but 
also tends to scale better. For example, eager translation suffices for 
our small database examples; larger-scale problems may require ef- 
ficient specialized data structures (e.g., indices), and thus mandate 
a background solver. 

For the rest of the paper, we focus on the lazy approach. A 
general framework for combining ILP with theories promises ad- 
vances in synthesis. We will therefore define an ILP Modulo Theo- 
ries (IMT) framework, and provide an abstract BC(r) architecture 
for lazy IMT solving. 

3. BC(r) 

In this section, we formally define IMT. We also provide a general 
BC(T) architecture for solving IMT problems. We describe BC(r) 
by means of a transition system, similar in spirit to the DPLL(r) 
transition system 1 36 1. It is intended to describe how to combine an 
ILP solver and a background decision procedure for a theory T to 
obtain a solver for ILP Modulo T. BC(r) models and generalizes 
(a) the branch-and-cut family of algorithms, (b) the lazy approach 
to Satisfiability Modulo Theories, and (c) the non-deterministic 
Nelson-Oppen combination of decision procedures, where one of 
the theories is Linear Integer Arithmetic. 

3.1 Formal Preliminaries 

An integer linear expression is a sum of the form Ci i)i + . . . + 
for integer constants Ci and variable symbols Vi . An integer linear 
constraint is a constraint of the form e t<l r, where e is an 
integer linear expression, r is an integer constant, and ix) is one 
of the relations <, <, =, >, and >. An integer linear formula is 
a set of (implicitly conjoined) integer linear constraints. We will 
use prepositional connectives over integer linear constraints and 
formulas as appropriate, and omit A when this does not cause 
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entity app { 
id: string; 
t_ram: int; 
t_cpu: int 

>; 

entity cabinet ■[ 
id: string; 
c_ram: int; 
c_cpu: int 

>; 

(a) Entities 



var to = app{"tO"; 64; 200000}; 
(* omitting apps tl through t6 *) 
var t7_0 = app{"t7_0"; 128; 200000}; 
var t7_l = app{"t7_l"; 128; 200000}; 

var cO = cabinet{"cO" ; 256; 1000000} 

var cl = cabinet{"cl" ; 256; 1000000} 

var c2 = cabinet{"c2" ; 256; 1000000} 

var c3 = cabinet{"c3" ; 256; 1000000} 

(b) Instances 



var apps = 

[tO; tl; t2; t3; t4; 
t5; t6; t7_0; t7_l] : 

var cabinets = 

[cO; cl; c2; c3] ; 

(c) Lists 



map m apps cabinets 
[t_ram, t_cpu] 
[c_ram, c_cpu] ; 

(d) Applications to Cabinets Map 



forall c in cabinets: 
m(t7_0, c) implies 
not m(t7_l , c) ; 

(e) Separation Constraint 



Figure 7: Avionics Architectural Synthesis Model 



ambiguity (i.e., juxtaposition will denote conjunction). An integer 
linear programming (ILP) instance is a pair C, O, where C is an 
integer linear formula and the objective function O is an integer 
linear expression. Our goal will always be minimizing the objective 
function. 

We assume a fixed set of variables V. An integer assignment A 
is a function V — )■ Z, where Z is the set of integers. We say that 
an assignment A satisfies the constraint c — (civi . . . CnVn ixi r) 
(where ix is one of the relations <, <, —, >, >, and every Vi is in 
V) if '^-Ci-A{vi) tx r. An assignment j4 satisfies a formula C if it 
satisfies every constraint c £ C. A formula C is integer-satisfiable 
or integer-consistent if there is an assignment A that satisfies C. 
Otherwise, it is called integer-unsatisfiable or integer-inconsistent. 

A signature E consists of a set S'^ of constant symbols, a set 
of function symbols, a set of predicate symbols, and a function 
ar : U — > that assigns a non-zero natural number 
(the arity) to every function and predicate symbol. A E-formula 
is a first-order logic formula constructed using the symbols in E. 
A E-theory T is a closed set of E-formulas (i.e., contains no free 
variables). When E is clear from the context, we will simply write 
theory in place of E-theory. 

Example 1. Let T,a be a signature that contains a binary func- 
tion read, a ternary function write, no constants, and no predicate 
symbols. The theory T4 of arrays (without extensionality) is de- 
fined by the following formulas HSOV : 

Va Vi Ve [read(write(a, i, e),i) — e] 

VaViVj Ve [i ^ j ^ read(write(a, i, e), j) — read(a,j)]. 

A formula F is T-satisfiable or T-consistent if F A T is satisfi- 
able in the first-order sense (i.e., there is an interpretation that sat- 
isfies it). A formula F is called T-unsatisfiable or T-inconsistent 
if it is not T-satisfiable. For formulas F and G, F T-entails G (in 
symbols F \=t G)if F A -iG is T-inconsistent. 

Definition 1 . Let T,z be a signature that contains constant sym- 
bols {0, ±1, ±2, . . .}, fl binary function symbol +, a unary func- 
tion symbol —, and a binary predicate symbol <. The theory of 
Linear Integer Arithmetic, which we will denote by Z, is the Tiz- 
theory defined by the set of closed T,z-,formulas that are true in the 



standard model (an interpretation whose domain is li, in which the 
symbols in Ez are interpreted according to their standard meaning 
overZ). 

We will use relation symbols like < that do not appear in E^, 
and also multiplication by a constant (which is to be interpreted 
as repeated addition); these are only syntactic shorthands. We will 
frequently view an integer assignment A as the set of formulas 
{v = A{v) \ V G V}, where A(v) is viewed as a E^ term. An 
integer assignment A viewed as a set of formulas is always Z- 
consistent. If A is an integer assignment and A satisfies an integer 
linear formula C, it is also the case that A \=z C.If A is an integer 
assignment, T is a theory and T is a formula, we will say that A is 
a T-model of F if ^ is T-consistent and A \— zut F. 

A T,-interface atom is a E-atomic formula (i.e., the application 
of a predicate symbol or equality), possibly annotated with a vari- 
able symbol, e.g., [x = y)". The meaning of the annotation is that 
the atomic formula has to hold iff the (integer) value of v is greater 
than 0; for our example we have x = y ^ v > Q. 

Definition 2 (ILP Modulo T Instance). An ILP Modulo (The- 
ory) T instance, where the signature E ofT is disjoint from T,z, is 
a triple of the form C, I, O, where: 

• C is an integer linear formula. 

• I is a set ofT,-interface atoms. 

• O is an objective function. 

The variables that appear in both C and I are called interface 
variables. 

An ILP Modulo T instance can be thought of as an integer linear 
program that contains terms that have meaning in T. In the above 
definition, the interface atoms (elements of /) are separated from 
the linear constraints, i.e., there are no E-terms embedded within 
integer linear constraints. This is not a restriction, because every 
quantifier- free (E U E^) -formula can be written in separate form 
(refer to the "Variable Abstraction" step of ||26il ). 

Example2. Let Y, be a signature that contains the unary function 
symbol f. The formula f{x + 1) + /(y + 2) > 3 (where x and y 
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are variable symbols) can be written in separate form as 

C = {vz + V4,>'i,vi = X 1,V2 = y + 2} 

I ^{V3^ f{vi), V4, = f(v2)} 

C is an integer linear formula; I is a set of Yi-interface atoms; 
and S is disjoint from T,z- Variable abstraction introduced new 
variables, vi, . . . ,V4,. C and I only share variable symbols. 

Let A be an assignment; A = {x = 2,y = l,Vi = 'i,V2 = 
3, W3 = 3, t;4 = 0}. Clearly A \=z C. However, A ^0 /, where 
stands for the theory of uninterpreted functions (also called the 
empty theory, because it has an empty set of formulas }. The reason 
is that Hi = «2 but f{vi) 7^ f{'V2)- In contrast, the assignment 
A' = {x — 2,y = l,vi — 3,V2 — S,V3 — 3,V4 — 3} is a 
0-model ofCM. 

3.2 Transition System 

Definition 3 (Simple Equality). A simple equality is a con- 



straint of the form Vi = c or Vi 



c, where Vi and Vj 



are integer variables and c is an integer constant. 

Definition 4 (Subproblem). A subproblem is a pair of the form 
(C, D), where C is a set of constraints and D is a set of simple 
equalities. 

A set of simple equalities resembles an assignment. Simple 
equalities and assignments differ, because the latter cannot contain 
equalities between variables with integer offsets. In a subproblem 
(C, D), we distinguish between the arbitrary constraints in C and 
the simpler constraints in D in order to provide a good interface 
for the interaction between the core ILP solver and background 
theory solvers that understand only a limited fragment of 2. It is 
the responsibility of the core solver to notify the theory solver about 
the equalities that hold. The motivation for equalities with offsets is 
to capture the interaction with congruence closure algorithms that 
take offsets into consideration |34|. Simple equalities are a special 
case of integer linear constraints. Thus, we will freely conjoin (sets 
of) simple equalities and other constraints. 

Definition 5 (State). A state of BC(T) is a tuple P II A, where 
P is a set of subproblems and A is either the constant None, or an 
assignment, possibly annotated with the superscript — cx3. 

Our abstract framework maintains a list of open subproblems, 
because it is designed to allow different branching strategies. This 
is in contrast to an algorithm like DPLL that does not keep track 
of subproblems explicitly. There, subproblems are implicit, i.e., 
backtracking corresponds to exploring a different subproblem. ILP 
algorithms branch over non-Boolean variables in arbitrary ways, 
thus mandating explicit subproblems. 

In a state P || A, the assignment A is the best known (T- 
consistent) solution so far, if any. It has a superscript —00 if it 
satisfies all the constraints, but is not optimal because the value 
of the objective function can be arbitrarily low. If this is the case, it 
is useful to provide an assignment but also to report the fact that no 
optimal solution exists. 

The interface atoms I and the objective function O are not part 
of the BC(r) states because they do not change over time. obj{A) 
denotes the value of the objective function O under assignment A: 
if O = X^i Ci^i, then obj{A) = Ci • A{vi). The objective 
function itself is not an argument to obj because it will be clear 
from the context which objective function we are referring to. For 
convenience, we define 06 j (None) = +00 andobj{A~°°) = —00. 
Function lb{{C, D}) returns a lower bound for the possible values 
of the objective function O for the subproblem (C, D): there is no 
A such that A satisfies C A D, and obj (A) < lb{{C, D)). 

Figure[8]defines the transition relation — > of BC(T) (a binary 
relation over states). In the rules, c and d always denote integer 



Branch 



Learn 



Forget 



Propagate 



PW{{C,D)} \\A^ 

PU{{Ci,D) I 1 < i < n} II A 
n> 1 

if I (C7^Vi<,<„C70 
I Ci are syntactically distinct 

PW{{C,D}} \\A^Pu{{Cc,D} \\A 
if CAD 1=2 c 

Pw{{Cc,D)}\\A^ PU {{C, D)} II A 
if CAD 1=2 c 

PW{{C,D}} \\A^Pu{{C,Dd)}\\A 
if CAD 1=2 d 



Drop 



PW{{C,D)} II A 



P II A 



if C A D is integer-inconsistent 



PW{{C,D)} \\A 



P II A 



Prune J None 

^ lb{{C,D))>obj(A) 



PW{{C,D}} \\A^ P\\A' 

A' is a T-model of C A D A / 

if 



Retire | obj {A') < obj (A) 



Unbounded 



T-Learn 



VD : if B is a T-model of C A D A /, 
then o6j(^') < obj{B) 

PW{{C,D)} II 4 II A'-°° 

!A' is a T-model of C A D A / 
obj{A') < obj{A) 
Vfc : 3B: B is a T-model of C A D A /, 
obj{B) < k 

PW{{C,D)}\\A — > PU {{C c, D) II A 
if 3F : C A D 1=2 T, F A / |=t c 

Figure 8; The BC(T) Transition System 



linear constraints and simple equalities. C (possibly subscribed) 
denotes an integer linear formula (set of integer linear constraints), 
while D denotes a set of simple equalities. C c stands for the 
set union C U {c}, under the implicit assumption that c ^ C; 
similarly for D d. C and D are always well-formed sets, i.e., they 
contain no syntactically duplicate elements. P and P' stand for sets 
of syntactically distinct subproblems, while A and A' are integer 
assignments. P tU Q denotes the union P U Q, under the implicit 
assumption that the two sets are disjoint. F stands for a (E U E2)- 
formula, where E is the signature of T. The intuitive meaning of 
the different BC(T) rules is the following: 

Branch 

Case-split on a subproblem (C, D), by replacing it with two 
or more different subproblems {d, D). If there is a satisfying 
assignment for C A D, this assignment will also satisfy d AD 
for some i, and conversely. 
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Learn, Propagate, T-Learn 

Add an entailed constraint (in the case of Learn and T-Learn) 
or equality (Propagate) to a subproblem. T-Learn takes the 
theory T into account. 

Forget 

Remove a constraint entailed by the remaining constraints of a 
subproblem. 

Drop, Prune 

Eliminate a subproblem because either it is unsatisfiable (Drop), 
or it cannot lead to a solution better than the one already known. 

Retire, Unbounded 

The solution to a subproblem becomes the new incumbent so- 
lution, as long as it improves upon the objective value of the 
previous solution. If there are solutions with arbitrarily low ob- 
jective values, we don't need to consider other subproblems. 

We define the binary relations — >^ and — >■* over BC(r) 
states as follows: S — >^ S' if S — > S', or there exists some state 
Q such that S — Q and Q — > S'. S — >* 5" if S = 5" or 
S — y'^ S' . When convenient, we will annotate a transition arrow 
between two BC(T) states with the name of the rule that relates 
them, for example S — > S' . 

BrEmch 

A starting state for BC(T) is a state of the form { (C, 0) } where 
C is the set of integer linear constraints of an ILP Modulo T 
instance. A final state is a state of the form || A (A can also 
be None, or annotated with -oo). If {(C, 0)} — || None, 
then the instance is T-infeasible, while if {{C, 0)} — ^+ || A 
for A 7^ None, we have found a T-model (which is optimal if an 
optimal solution exists). 

In the subsections that follow, we provide soundness and com- 
pleteness proofs for BC(r) (Theorems [T] |2] and [5](. We strive to 
provide a general framework that can encompass a wide range of al- 
gorithms; the BC(r) transitions are thus highly non-deterministic. 
Subsection |3.6| provides guidelines for a practical implementation. 

3.3 Soundness 

Throughout this section, we assume an IMT instance with objective 
function O and a set of interface atoms /. 

Lemma 1. For states P \\ A and P' \\ A' such that P \\ A — ¥ 
P' II A' , if there is an assignment B such that B is a T-model 
of C A D A I for some {C, D) G P, then one of the following 
conditions holds: either (i) obj(A') < obj{B), or (ii) B is a T- 
model ofC A D' A I for some {C',D') G P' . 

Proof. Assume that there is a subproblem p — {C,D) £ P 
and an assignment B such that B is a T-model of C A D A I. 
If p G P', then ^ holds trivially. If p ^ P', then the transition 
cannot possibly be Drop (we cannot apply Drop on (C, D) because 
B is an assignment that satisfies C A D). For the other rules: 

• Branch: There are subproblems (Ci, D), . . ., (C„, D) in P' 
such that D {C ^ Vi<i<n (^i)- Thus, B is a T-model of 
C, A D A / for some i (1 < i< n). {Ct,D} £ P', therefore (jiij 
holds. 

• Prune: obj(A') < lb{C A D) < obj{B). Condition JJ holds. 

• Retire, Unbounded: obj{A') < obj{B), therefore (li]l holds. 

• Learn, Forget, Propagate, T-Learn: there is a subproblem 
{C',D'} G P' such that C A D A I ^zut C A D' , and 
therefore B is a T-model of C" A D' A /. 



Proof. The only rules that modify the assignment are Retire and 
Unbounded; the conditions under which we can apply them imply 
obj{A') < obj{A). For any other rule, A ^ A' . 

Lemma 3. For states P \\ A and P' \\ A' such that P \\ A — > 
P' \\ A'.ifA^ A' then A' is a T-model ofC AD Alfor some 

{C,D)eP 

Proof. The conditions of Retire and Unbounded guarantee that 
A' is a T-model of C A D A / for some (C, D) G P. No other rule 
modifies the assignment. 

Lemma 4. For states P \\ A and P' \\ A' such that P \\ A — ¥* 
P' II A', if there is an assignment B such that B is a T-model 
of C A D A I for some {C\ D) G P, then one of the following 
conditions holds: obj{A') < obj{B), or B is a T-model ofC' A 
D' A I for some {C',D') G P'. 

Proof. We induct on the length n of the sequence of transitions. 

• Induction base: n = 0. P \\ A = P' \\ A'; obvious. 

• Induction step: assume that the property holds for any sequence 
of n — 1 transitions, where n > 1. We will prove that it 
holds for any sequence of transitions Pq || Aq — > ... — > 
Pn-i II An-i — > Pn \\ A,i . Assume there is an assignment B 
such that S is a T-model of Co A Do A I for some (Co, T'o) G 
Po- By the induction hypothesis, one of the two following 
conditions holds: 

■ obj{An-i) < obj{B): then obj{A„) < o6j(A„_i) < 
obj{B) from lemma [5] 

■ i? is a T-model of C„_i A Dn-i A I for some subprob- 
lem (Cn-i, -Dji-i) G Pn-i'- our proof obligation follows 
from lemma [T] applied to the transition Pn-i \\ A„-i — > 

Pn II An. 

Lemma 5. For states P \\ A and P' \\ A' such that P \\ A — > 
P' II V(c,D)eP' CAD^z V(c,i5)ep C A D. 
Proof. We case-split on the BC(T) transitions. 

• The rules Prune, Drop, and Retire can only make the disjunc- 
tion of the subproblems in P' stronger, because a subproblem 
is eliminated and the rest of the subproblems remain intact. 



For Unbounded, \J 



,C A D = false; false \=i 



• The rules Learn, Forget, and Propagate and substitute a 
subproblem for a ^-equivalent one. 

• The rule T-Learn adds a constraint to a subproblem, and there- 
fore makes the disjunction in P' stronger. 

• The rule Branch replaces a subproblem (C, D) with a set of 
subproblems whose disjunction is ^-equivalent to C A T>. 

Lemma 6. For states P \\ A and P' || A' such that P 



A 



P' M',V(c,D)eP'CADh^ V 



CAD. 



Lemma 2. For states P \\ A and P' 
P' II A', obj{A') < obj{A). 



A' such that P W A 



Proof. Induction on the length of the sequence of transitions and 
application of lemma|5] 

Theorem 1. For aformulaC, i/{(C,0)} II None — || None, 
then C A I is T -unsatisfiable. 

Proof. Assume that there is an assignment A such that A is a T- 
model of C. Then, from lemma |4] either there exists (C', D') G 
such that A is a T-model of C" A D' A / (which cannot possibly 
be true), or +oo = obj {None) < obj{A) (contradiction, because 
obj{A) has to be finite). 

Theorem 2. For a formula C and an assignment A, if 
{{C,0)}|1 None ^* || A 
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where A 7^ None, then (a) A is a T -model ofC A I, and (b) there 
is no assignment B such that B is a T-modet of C A I and 
obj{B) < obj{A). 

Proof, (a) The sequence of transitions from {{C, 0)} || None to 
II A has to be of the following form: 

{{C,0)} II None ^* Si Ml — ^ 

Retire /Unbounded 

S\\A^* 0||yl 

There is at least one Retire or Unbounded step, as these are 
the only rules that can modify the assignment. Consider the last 
such step. The conditions on Retire and Unbounded steps require 
that A is a. T-model of I A V{c o>eSi ('^ ^ From lemma |6] 
y{c,D}eSt (C A D) j=z C. Therefore! A is a T-model of C. 

(b) Follows from lemma[4] 

3.4 Termination 

An algorithm that implements the transition system in Figure [8] 
has flexibility with respect to branching: in particular, a non- 
terminating branching strategy is possible. In addition, an infinite 
number of ILP inequalities is implied by any arbitrary set of ILP 
constraints, and they can be added to the problem by means of the 
Learn rule, or forgotten by means of Forget. To ensure termi- 
nation, we impose constraints on how the rules Branch, Learn, 
Forget, and T-Learn can be applied. 

Theorem 3 (Termination). A sequence of BC(T) transitions is 
finite if and only if the following conditions hold: 

(a) The sequence has a finite number o/Branch steps. 

(b) Every infinite suffix of the sequence includes a step that is not 
one o/Learn, Forget, and T-Lea.rn. 

Proof. The only if direction is obvious. Suppose a sequence a 
satisfies (jajl and ijbj. Then, by |a|(, it has a suffix r with no Branch 
steps. The number of subproblems in r never increases (that would 
require a Branch step), and decreases whenever we perform a 
Drop, Prune, Retire, or Unbounded step. Thus, r can only con- 
tain a finite number of Drop, Prune, Retire, and Unbounded 
steps. Therefore, r has a suffix, ^, without any Drop, Prune, 
Retire, and Unbounded steps. Now ^ can only contain a finite 
number of Propagate steps, because we can learn at most one 
simple equality for each pair of variables in a subproblem. So ^ has 
a suffix, with only Learn, Forget, and T-Learn steps, but by 
condition ijbJ has to be finite. Therefore a is finite. 

3.5 Completeness 

Definition 6 (Stably Infinite Theory). A E-theory T is called 
stably infinite if for every T-satisfiable quantifier-free Ti-formula 
F there exists an interpretation satisfying FAT whose domain is 
infinite. 

Definition 7 (Arrangement). Let E be an equivalence relation 
over a set of variables V .The set 

a{V, E)={x = y\ xEy} U 

{x ^ y \ x,y £V and not xEy} 

is the arrangement of V induced by E. 

Note that Z isa stably infinite theory. We build upon the follow- 
ing result on the combination of signature-disjoint stably infinite 
theories: 

Theorem 4 (Combination of Stably Infinite Theories |26 |). Let 

Ti be a stably infinite T^i-theory, fi>r i — 1,2, and let Ei n E2 = 0. 



Also, let Ti be a conjunction ofEi literals. Ti U r2 is (Ti U T2)- 
satisfiable iff there exists an equivalence relation E of the variables 
shared by Fi and V2 such that Ti U a[V, E) is Ti-satisfiable, for 
i = 1,2. 

The theorem above implies that the combination of Ti = Z 
and another stably infinite theory is decidable: we can pick an 
arrangement over the variables shared by the two sets of literals 
non-deterministically and perform two T -satisfiability checks. We 
show how BC(r) can meet the hypotheses of the theorem to ensure 
completeness. 

Theorem 5 (Completeness). BC(T) provides a complete opti- 
mization procedure for the ILP Modulo T problem, where T is a 
decidable stably infinite theory. 

Proof Sketch. Let C, J, O be an ILP Modulo T instance. As- 
sume that 

(C,0) II None — y* P II A, 

and that for every (C, D) £ P the following conditions hold: 
(a) there is an equivalence relation E(^c,d) over the set of interface 
variables V of the ILP Modulo T instance, such that C A D \=z 
a{V, E(c,D)), and {h) C A D v > Q, or C A D v < Q 
for every v that appears as the annotation of an interface atom in /. 
Then we can solve the IMT instance to optimality as follows. For 
every subproblem (C, D) G P, we decide the T instance: 

{t\t £ I and t is not annotated} U 

{t\e e I &MC A D ^z V > Q} \J 
{-^t I G / and C A D ^2 u < 0} U 
a{V,Ei^c,D)) 

If the instance is T-unsatisfiable, then C A D A / is T-unsatisfiable. 
If it is T-satisfiable, any integer solution for C A D will be a T- 
model. For the T-unsatisfiable subproblems, we apply T-Learn 
to learn an integer-infeasible constraint (always possible because a 
T-inconsistent formula implies anything, e.g., < 0) and subse- 
quently apply Drop. If all the subproblems are T-unsatisfiable, we 
reach a final state || None. If there are T-satisfiable subproblems, 
it suffices to apply a complete algorithm for obtaining integer solu- 
tions (e.g., Gomory's method 1 17|) as we have already established 
T-consistency. The basic steps of such algorithms (for instance, 
adding a cutting plane) can be described by means of BC(T) steps. 
If the BC(T) sequence adheres to the hypotheses of Theorem|3] it 
will be finite. Also, note that objective functions do not hinder com- 
pleteness: it suffices to recognize an unbounded subproblem (T) and 
apply Unbounded. 

It is easy for an implementation of BC(T) to guarantee that 
after a finite number of steps every subproblem will entail an 
arrangement of the interface variables. For every pair of interface 
variables x and y and every subproblem, we apply the Branch rule 
to obtain three new subproblems each of which contains one of the 
constraints x < y, x = y, and x > y. Similarly, we can branch 
to obtain a truth value for w > for every v that appears as the 
annotation of an interface atom. 

3.6 Discussion 

A straightforward implementation of BC(T) is a branch-and-cut 
algorithm for ILP (branching and learning entailed constraints with 
cutting plane techniques) that communicates with theory solvers. 

Practical branch-and-cut algorithms rely heavily on continuous 
relaxations of the subproblems. Although BC(T) does not explic- 
itly make use of relaxations, it is designed to facilitate their use: 
(a) The lower-bounding procedure lb can rely on solutions to con- 
tinuous relaxations, as the best integral solution can be at most 
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as good as the best non-integral solution, (b) Solutions to relax- 
ations can guide the use of the Branch rule: e.g., if a solution as- 
signs a non-integer value r to variable v, we can branch on whether 
V > \r] or V < [rj . (c) If the relaxation of a subproblem is infea- 
sible, then the subproblem itself is infeasible; we apply the Drop 
rule, (d) If the relaxation of a subproblem has an T-consistent inte- 
ger solution, we can apply Retire. 

The BC(r) architecture does not preclude specialized tech- 
niques for special classes of linear constraints. For example, an 
implementation can use the two-watched-literal scheme to accel- 
erate Boolean Constraint Propagation on clauses. Our goal is not to 
replace propositional reasoning, but rather to shift a broader class 
of constraints from background theories to the core solver. 

4. SMT as IMT 

IMT can be thought of as an extension to SMT, i.e., an IMT solver 
can be used in place of an SMT solver. To support this claim, 
we show how to handle propositional structure within IMT, and 
thus solve SAT Modulo Z problems. We also prove that a version 
of the BC(r) transition system can simulate DPLL(T) |36|. A 
simulation in the reverse direction is impossible, because BC(r) 
natively handles optimization, while DPLL(T) does not. 

4.1 Handling Propositional Structure 

Clauses can be thought of as a special case of linear constraints: 
i.e., a clause ViZi is equivalent to T^ik > 1 (for translating a clause 
to a linear expression, a negative literal ^Vi appears as 1 — Vi 
while a positive literal remains intact). Thus, we can translate the 
propositional structure in an SMT instance as an equivalent set of 
linear constraints. 

SAT Modulo Z instances contain Z theory atoms under the 
propositional connectives. For every such atom t, we define an 
integer variable v{t) such that v{t) <^ t. It suffices to show 
how this works for linear inequalities of the form t = a ■ 

Vi < r); other relations can be expressed in terms of < and the 
propositional connectives. Assume that there are constants m, k 
such that m < J]]; Ci ■ Vi < k always holds. There exist such 
m, k if all variables are bounded (we can always impose bounds 
that preserve satisfiability |40|). The direction v{t) => t can be 
expressed as Ci- Vi < r + (fc — r) ■ (1 — v{t)); for the opposite 
direction we have Ci ■ Vi > r + (m — r) ■ v{t) . Subsequently 
v[t) appears in clauses in place of t. 

The technique we just described relies on computing variable 
bounds which can be of cosmic magnitude, but this is only a 
minor concern. In the worst case, large bounds will lead to weak 
continuous relaxations; the relaxations will become stronger once 
we start branching on the Boolean variables. Modern ILP solvers 
provide an alternative called indicator constraints^ i.e., natively 
supported constraints of the form u = 1 => J]] . Ci ■ Vi < r, where 
u is a {0, 1} variable. BC(r) does not explicitly provide indicator 
constraints, in order to stay within the standard formulation of ILP. 

4.2 Simulating DPLL(r) 

A DPLL(r) assignment is a sequence of literals that does not 
contain both a literal and its negation. Some literals are annotated 
as decision literals (written as /'*). A DPLL(r) state is either the 
special state FailState, or a tuple of the form M \\ F, where M 
is a DPLL(T) assignment and F is a propositional formula (set of 
clauses). For precise definitions and transition rules refer to |36|. 
To avoid confusion with BC(T), we denote the DPLL(r) transition 
relation by — >d. 

We add an extra rule (called Subsume) to the BC(T) transition 
system. We refer to the resulting transition system as BC(T)"^, and 



denote the BC(r)^ 



Subsume 



transition relation by — >/. 

P^{{C,D),{C',D')}\\A^ 
P\J{{C,D)} II A 

if Cad' "^zC ^D 

Intuitively, Subsume drops a subproblem {C',D') if there is a 
more general subproblem {C,D), e.g., if {C',D') was derived 
from (C, D) by making some decisions. Subsume is not part of 
the BC(r) system as described in Section [3] because we do not 
have evidence that such a rule would be useful for branch-and-cut 
algorithms. The sole purpose of Subsume is to help us simulate 
the DPLL(r) T-Backjump and Restart rules. We would not need 
Subsume if we were to simulate a version of DPLL(r) with a 
backjumping rule more restrictive than T-Backjump (e.g., jumping 
right above the highest T-inconsistent decision level), and without 
a Restart rule. 

The soundness proof of Section [3] can easily be adapted for 
BC(r)^. Uncontrolled Subsume poses an extra threat for termi- 
nation; the hypotheses of Theorem [5] have to be strengthened ac- 
cordingly. 

Definition 8 (Simulation Function R). 
7?(FailState) = || None 

RiM II F) = {([F], [M])} U L(M,F) \\ None. 

where 

L(M,F) = {IF],0)U 

{(IFim) \ 31 -.Nl" prefix of M} 

By J-] we denote the translation from a DPLL(r) clause (or 
formula) to a BC(r) linear inequality (or set thereof). We have 
outlined how the translation happens in Subsection |4.1| A positive 
DPLL(r) literal viewed as a simple equality is of the form v = 1 
(for a negative literal we have v — 0). The basic idea behind the 
simulation function R is that the negation of each DPLL(T) deci- 
sion is like a subproblem yet to be explored. DPLL(r) does not ex- 
plicitly represent subproblems, as they can be easily reconstructed 
from the assignment. 

Theorem 6 (Simulation). For DPLL(T) states S and S" such 
thatS — >D S', R{S) — >+ R{S'). 

Proof. We case-split on the DPLL(r) transition rules. 

Decide: S = M \\ F and S" = M / || F where M is an 
assignment, F is a formula, and / is a literal such that I or 
occurs in F, and I is undefined in M. 

R{S) = {([F], [M]}} U L(M, F) II None 

,[AfI>,([FI,[MI>}UL(M, F) II None 



-^i {m A I 

Branch 

-^I {(MAI 

Propagate 



[A/I A W>, ([FI, [A/I>} U L{M,F) || None 
{IF\,W\)}^ L{M, F) II None 



http://j .mp/NdkZZll(CPLEX);'http://j .mp/NdlmDs (SCIP) 



Forget 

= {([FI,[M/i>,([FI,[MI>}UL(A/,F) || None 
= {([-PI: W >■%} U L{M l'^, F) II None 
= H(S') 

Fail: 5" = M |i F, C and S" = FailState, where M ^ 
-^C and M contains no decision literals. L{M, (F, C)) = 0. 
fF, C] A fM} is ^-inconsistent, because M |= (where \= 
denotes DPLL(T) propositional entailment). 

R{S) = {(IF, C], IM])} II None -^j || None = R{S') 

Prune 
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UnitPropagate: S = M \\ F,C y I anA S' = M I \\ F,C y 
I where M \= -^C and / is undefined in M. Notice that 

L[M,F) = Lmi),F). 

R{S) = {(IF, C V /], IM])} U L{M, F) \\ None 
-^i {{IF,CVZ],IM;])}UL(M,F) II None 

Propagate 

= R{S') 

T-Propagate: S = M\\ F, and 5" = M I \\ F where M |=t I, 
I or -^l occurs in F, and I is undefined in M. Notice that 
iFim Mand P/],/Nt PJ. 



RiS) 



— >i 

T-Learn 



{(M,IA//])}uL(Af,F) II None 
{(IF]ApI,IMI)}uL(M,F) II None 

{(Ma P],IMZ])}UL(M,F) II None 



Propagate 

-^i{m,lMq)}uL{M,F) 

Forget 



None 



= R{s') 

T-Backjump: S ^ M I'' N \\ F,C and S" = M T || F, C where 
A/ l"^ N 1= ^C, and there is some clause C" V such that 
F, C !=T C" V and M \= ^C", is undefined in M, and I' 
or ^Z' occurs in F or in Af I''' N. 

R(S) = {{IF, CI, [M i'* AT]}} U L((M Z'* TV), (F, C)) || None 



Drop 



Subsume 



>I L{{M f N), (F,C)) li None 



{([F CI, IM})} U L(Af, (F C)) II None 



-^l {([FC,/'I,[A/I>}UL(A/,(FC)) II None 

T-Learn 

-^l {(IF, C, I'l IM I'D} U L(A/, (F C)) II None 

Propagate 

{([FCI,[A//'I)}UL(Af,(FC)) II None 

Forget 

= {{IF, d IM I'})} U L((A/ «'), (F C)) II None 
= R{S') 

T-Learn: 5 = M \\ F and S" = A/ || F, C. All subproblems 
in -R(5') have the same linear constraints, JF]; we can apply 
the T-Learn rule to each subproblem and learn JC]. Thus, 
R{S) — >+ R{S'). 

Learn 

T-Forget: similar to T-Learn: R{S) — >f R{S'). 

Forget 

T-Restart: 5" = Af || F and S" = || F. 



RiS) 



Subsume 



{(M,M)}UF(A/,F)||None 
{([F1,0)}|1 None 

: RiS') 



The simulation sometimes performs multiple BC(T) transitions 
for a single DPLL(T) transition. This is because we have to perform 
a certain action for multiple BC(T) subproblems (as many as the 
number of decision literals in the DPLL(T) assignment). To put 
Theorem [6] into perspective, BC(T) subproblems are an abstract 
construct, and do not necessarily resemble the data structures used 
by a practical implementation that branches in a controlled way, 
like DPLL(T). Conversely, the number of steps that the simulation 
requires is not to be interpreted as indicative of the performance of 
an IMT solver. 

5. Experiments 



As we described in Section |2.1.3[ our approach for synthesiz- 
ing architectural models combines an ILP solver (which handles 



1000 




10 100 
CPLEXTime [s) 



1000 



Figure 9: Z3 (best custom encoding) against CPLEX (no encoding) 



structural and resource constraints) with a decision procedure for 
scheduling. We now provide experimental evidence that an ILP 
solver is essential for scalability, i.e., a SAT-centric combination 
of decision procedures (SMT) would not work. 

To evaluate SMT against our resource and structural require- 
ments, we perform the following experiment: we run CoBaSA 
on a characteristic 787-derived instance and record all ILP solver 
queries (21 in total). The instances are essentially pseudo-Boolean 
(PB) problems, i.e., they consist of linear constraints over Boolean 
(or {0, 1}) variables. The ILP solver we use is CPLEX, a state- 
of-the-art industrial solver. The SMT solver we use is Z3 1101 . 
the QF_LIA (Quantifier-Free Linear Integer Arithmetic) winner of 
SMT-COMP201lE] 

In our first experiment, we compare CPLEX with Z3, using the 
obvious encoding: we generate conjunctions of linear constraints, 
using integer variables bounded from to 1. CPLEX is given the 
constraints using its format and Z3 is given constraints in the SMT- 
LIB 2.0 format. With a timeout of 10 hours, Z3 fails to provide a 
solution for any of the instances. In contrast, CPLEX solves all in- 
stances, requiring an average of 16 seconds per instance. Therefore, 
CPLEX outperforms Z3 by at least four orders of magnitude. 

In our next experiment, we tried numerous encodings to see 
if we could trade human time for improved Z3 performance. We 
report on the best we were able to accomplish. Observe that the 
variables in the above instances are in fact Boolean. Some of 
the linear constraints are clauses, i.e., of the form "^^^i ^ 1 
for literals h. It makes sense to help Z3 by encoding such con- 
strains as clauses. To do this, we declare all variables to be 
Boolean. Since almost all variables also appear in arithmetic 
contexts where they are multiplied by constants greater than 1, 
we translate such constraints as demonstrated by the following 
example: the linear constraint x + y + 2- z > 2 becomes 
(>= (+ (ite X 1 0) (ite y 1 0) (ite z 2 0)) 2). With 
this encoding, Z3 solves all problems, but performance is 36 
times slower than CPLEX on average. Figure |9] visualizes the per- 
formance of Z3, using the best encoding we could find, versus 



See http : / /www . smt- comp . org In fact, Z3 won most other categories. 
It also outperformed all other solvers in all arithmetic categories of SMT- 
COMP 2012, although the previous winner does not officially compete. 
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CPLEX, using the default encoding (for which Z3 provided no so- 
lutions). Notice that even with human intervention, Z3 is between 
1 and 2 orders of magnitude slower than CPLEX for most of the 
problems. 

Our last SAT-based option is to reduce the structural and archi- 
tectural requirements to propositional SAT, and bypass SMT and 
QF_LIA altogether. In essence, we have to translate PB constraints 
to SAT. We explored this option in depth |27|. In summary, we 
did manage to approach the performance of ILP, but it required 
a custom PB solver that combines incremental translation to SAT 
with reasoning at the PB level; eager translation followed by SAT 
solving was orders of magnitude worse. For the sequence of 21 
instances, our PB solver is four times slower than CPLEX. 

So far our experimentation has focused on feasibility problems, 
while in reality commercial ILP solvers like CPLEX are primar- 
ily developed for optimization. There is no wide consensus in the 
SAT and SMT community on how to best perform optimization. 
Relatively simple techniques are applied, e.g., decrement-by-one. 
As we argued previously, the use of objective functions is of fun- 
damental importance to synthesis problems. When we also con- 
sider optimization, this tends to significantly amplify the perfor- 
mance gap between ILP and SAT-based solvers (in favor of ILP 
solvers). To highlight this, we perform one last experiment. We 
encode the fiindraising problem in CoBaSA: 56 guests and their 
spouses (78 people in total) are invited to a dinner. Some of the 
guests are Northeastern University (NEU) officials, while the rest 
are supporters of the university. The guests have to be mapped to 
8 tables of 10 seats. We obviously have resource constraints for 
seats. Each guest consumes one seat, unless they bring along their 
spouse, in which case they consume two seats. There are separa- 
tion constraints, because some supporters dislike each other. The 
objective is to maximize the number of supporters co-located with 
NEU officials (the latter are planning to ask for contributions). The 
problem is moderately hard, as it takes 9 seconds for CPLEX to 
find an optimal allocation. Our PB solver finds a feasible allocation 
in less than a second; however it takes close to two hours to find a 
provably optimal solution. 

Finally, it is worth pointing out that evaluation of ILP solvers 
against SAT-inspired engines with respect to optimization takes 
place yearly, as part of the PB Competition|^ There, CPLEX and 
SCIP m dominate the SAT-based algorithms. 

6. Related Work 

6.1 Nelson-Oppen, SAT, and SMT 

The seminal work of Nelson and Oppen [321 provided the foun- 
dations for combining decision procedures for different theories. 
Tinelli and Harandi |41 1 revisit the Nelson-Oppen method and pro- 
pose a non-deterministic variant that works for non-convex stably 
infinite theories. Manna and Zarba provide a detailed survey of 
Nelson-Oppen and related methods |26|. 

ILP Modulo Theories resembles Satisfiability Modulo Theories, 
with ILP as the core formalism instead of SAT. SMT has been 
the subject of active research over the last decade (e.g., Isl 1111 
|36|). Nieuwenhuis, Oliveras and Tinelli |36| present DPLL(T), an 
abstract framework that can be used to formally reason about the 
lazy SMT technique. Based on DPLL(r), one can obtain a solver 
for satisfiability modulo a theory T by simply plugging in the 
corresponding theory solver SolverT. The SMT framework has 
been extended to deal with optimization problems (8I I35II391 . 

Different fragments of Linear Arithmetic have been studied 
as background theories for the SMT framework. Dutertre and de 
Moura H2J present a Simplex-based Linear Arithmetic solver for 



* See http : //www. cril .univ-artois . f r/PB12/ 



DPLL(T'). The same authors employ a branch-and-cut strategy to 
handle integer or mixed integer problems |13|. Overall, Linear 
Arithmetic as a background theory for SMT differs significantly 
from IMT, because in our case the core solver is an ILP solver. 

A family of linear solvers that apply ideas from SAT has re- 
cently appeared 1211 12411381 . These algorithms essentially general- 
ize DPLL-style search for a satisfying assignment to non-Boolean 
variables, and can be thought of as steps towards SMT with a non- 
propositional core engine. Our research direction is complemen- 
tary, as we do not focus on the core solver, but rather provide a 
general framework for combining ILP with theories. 

6.2 Linear Programming 

Branch-and-cut algorithms 1311 combine branching (branch-and- 
bound) with cutting plane techniques: adding violated inequalities 
(cuts) to the linear formulation. Different methods have been stud- 
ied for generating cuts for general integer programming problems, 
starting with the seminal work of Gomory |17|. Cuts can also be 
generated in a problem-specific way, e.g., for TSP Il8l . Problem- 
specific cuts are analogous to theory lemmas in our framework. 

Another family of linear programming techniques that bears re- 
semblance to IMT is Benders decomposition |6|. The linear pro- 
gramming problem is split into a master problem (which has only 
a subset of the original variables) and a subproblem. The master 
problem is solved first, and then the subproblem is solved with the 
values of the master problem fixed ("trial" values). If the "trial" 
values are unacceptable, a cut is derived and added to the master 
problem. Logic-based Benders Decomposition |20| generalizes the 
strategy so that the master problem, the subproblem, or both are not 
necessarily linear programs. In IMT, the problem is "decomposed" 
into a core ILP instance and background theory problems. 

Linear and integer programming solvers generally perform 
floating-point (FP) and thus inexact calculations. Faure et al. ex- 
periment with the inexact CPLEX solver as a theory solver 1151 
and observe wrong answers. For many applications, numerical in- 
accuracies are not a concern, e.g., the noise in the model overshad- 
ows the floating point error intervals, or an answer close enough 
to the theoretical optimal suffices. However, accuracy is often crit- 
ical. Recent work |9 33] proposes using FP arithmetic as much 
as possible (especially for solving continuous relaxations), while 
ensuring that cutting planes, infeasibility certificates, and bounds 
obtained from relaxations are safe. IMT solvers can be built on top 
of both exact and inexact solvers, depending on the requirements. 

6.3 Languages 

We are not the first to pursue the idea of an extensible language. 
Lisp and Scheme implementations have long provided powerful 
macro systems [14, 23, among many others]. The modern-day 
Racket platform provides an arsenal of extension mechanisms 1 42] 
for defining DSLs and hosting different paradigms. CoBaSA is 
a specification language as opposed to a programming language; 
thus, the ways to achieve extensibility differ, e.g., we need infras- 
tructure for combining decision procedures. 

Modeling a problem directly as a linear program can be difficult 
and error-prone. Higher-level modeling languages like AMPL 1371 . 
GAMS 1 3 1, and Zimpl |22 | aim at making optimization technology 
more accessible, an objective that they share with CoBaSA. Unlike 
these languages, we allow the programmer to extend the underlying 
formalism by means of theories, and expose theories to the high- 
level language. 

7. Conclusions and Future Work 

We introduced the ILP Modulo Theories (IMT) framework for de- 
scribing problems that consist of linear constraints along with back- 
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ground theory constraints. We did this via the abstract BC(T) tran- 
sition system that captures the essence of the branch-and-cut family 
of algorithms for solving IMT problems. We showed that BC(r) 
is a sound and complete optimization procedure for the combina- 
tion of ILP with stably-infinite theories and that it generalizes the 
SMT framework. Our first IMT-inspired tool, CoBaSA, has been 
used to solve hard synthesis problems arising in the design of Boe- 
ing's 787. Our experience indicates that the IMT framework cou- 
pled with high-level modeling languages like CoBaSA has great 
potential for solving hard, industrial synthesis problems. 

Many interesting research directions now open up. Among the 
more theoretical directions, it would be interesting to explore how 
we can relax the stably-infinite requirement on the background the- 
ory and still guarantee soundness and completeness. Our expecta- 
tion is that there are fruitful connections between IMT and other 
formalisms and techniques, e.g., SMT, constraint programming, 
domain-specific cut generation, and decomposition. For example, 
we believe that many instances of these techniques can be described 
more simply and elegantly using IMT. 

The more practical research directions include implementing 
and experimenting with a general IMT solver based on BC(r). This 
work is underway. We are planning to provide an SMT-LIB fron- 
tend, so that IMT can be used in place of an SMT solver; we out- 
lined a strategy for mimicking SMT in Section]?] Future versions of 
CoBaSA will use our BC(r)-based solver as the backend. This will 
allow us to provide powerful, theory-based CoBaSA DSLs, e.g., a 
comprehensive DSL for relational algebra, supported by a solver 
that uses efficient database techniques. A possible application is 
IMT-based SQL optimization. 
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